Is Sharma-Mittal entropy really a step beyond Tsallis and Renyi entropies? 
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We studied the Sharma-Mittal relative entropy and showed that its physical meaning is the free 
energy difference between the off-equilibrium and equilibrium distributions. Unfortunately, Sharma- 
Mittal relative entropy may acquire this physical interpretation only in the limiting case when both 
parameters approach to 1 in which case it approaches Kullback-Leibler entropy. We also note 
that this is exactly how Renyi relative entropy behaves in the thermostatistical framework thereby 
suggesting that Sharma-Mittal entropy must be thought to be a step beyond not both Tsallis and 
Renyi entropies but rather only as a generalization of Renyi entropy from a thermostatistical point 
of view. Lastly, we note that neither of them conforms to the Shore- Johnson theorem which is 
satisfied by Kullback-Leibler entropy and one of the Tsallis relative entropies. 
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I. INTRODUCTION 



Recently there has been a growing interest in generalized entropies such as Tsallis [1] and Renyi [2] entropies in 
the context of a generalized thermostatistics. For example, Tsallis entropy has been applied to nonlinear diffusion 
equations [3] and Fokker-Planck systems [4, 5] while Renyi entropy has been studied for its inverse power law equi- 
librium distribution [6] and has been shown to satisfy the zeroth law of thermodynamics [7]. It has been observed 
that an unifying entropy exists which seems to generalize both of these entropies [8, 9, 10]. This two-parametric 
entropy is called Sharma-Mittal entropy [11]. It generalizes Tsallis, Renyi and Boltzmann-Gibbs (BG) entropies since 
its two parameters generate these entropies in limiting cases. Our aim in this paper is to shed some light on this 
new entropy measure from a thermostatistical point of view. It is organized as follows: In Section II, the review 
of the physical meaning of BG relative entropy i.e., Kullback-Leibler entropy will be given. It will be seen that it 
provides a measure of free energy differences between the off-equilibrium and equilibrium distributions. In Section 
III, we will briefly mention Sharma-Mittal entropy and some of its important properties. In Section IV, we will study 
the relative entropy associated with this entropy and show that it can be given a physical meaning in a generalized 
thermostatistical framework in terms of free energy differences only when Sharma-Mittal relative entropy reduces to 
Kullback-Leibler entropy. Its connection with Renyi relative entropy will be investigated and shown to behave in a 
similar manner. Lastly, in Section V, we will see that Sharma-Mittal entropy does not conform to Shore-Johnson 
theorem sharing also this feature with Renyi relative entropy. We will summarize the conclusions in Section VI. 



II. THE PHYSICAL MEANING OF KULLBACK-LEIBLER ENTROPY 

The relative entropy is an important concept whose uses range from the numerical analysis of protein sequences [12], 
pricing models in the market [13], to medical decision making [14]. In this paper, we will study it from a thermosta- 
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tistical point of view. In order to study the physical meaning of any relative entropy in a thermostatistical framework, 
one has first to obtain the equilibrium distribution associated with the entropy of that particular thermostatistics. 
This can be done by maximizing BG entropy subject to some constraints following the well known recipe of entropy 
maximization. BG entropy reads 



w 

Sbg(p) = -^Pjlnpi, (1) 

i 

where pi is the probability of the system in the ith microstate, W is the total number of the configurations of the 
system. Note that Boltzmann constant k is taken to be equal to one throughout the paper. Let us assume that the 
internal energy function is given by U — £ iPi where e» denotes the energy of the ith microstate. In order to get the 

i 

equilibrium distribution associated with BG entropy, we maximize the following functional 



w WW 
= ~^2Pi ln Pi- a ^2Pt- P^SiPi, (2) 

i it 

where a and f3 are Lagrange multipliers related to normalization and internal energy constraints respectively. 
Equating the derivative of the functional to zero, we obtain 

= -lnfc-1 = 0. (3) 

5pi 

Tilde denotes the equilibrium distribution obtained by the maximization of BG entropy. By multiplying Eq. (3) 
by Pi and summing over i, using the normalization and internal energy constraints, we have 

a + 1 = S BG - 0. (4) 
Substitution of Eq. (4) into Eq. (3) results in the following equilibrium distribution 



Pi =e- SBG e l3U e~P £ \ (5) 
The relative entropy in ordinary BG case is called Kullback-Lcibler (K-L) entropy [15]. It reads 

K\p\\q] = J2PiHPi/<li)- (6) 

Note that it is a convex function of Pi, always non- negative and equal to zero if and only if p = q. If we now use 
the equilibrium distribution p as the reference distribution in K-L entropy, we can write 

K\p\\p\=J2pMPi/Pi)- (7) 

i 

The equation above can be rewritten as 

K\p\\p\ = -S BG -Y J Pi^Pi- (8) 

i 

We then insert the equilibrium distribution given by Eq. (5) in the equation above to find 



K[ P \\p\ = S BG - ]Tp 4 (-S BG + I3U- fa). 

i 



(9) 
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Taking care of the effect of summation, we have 

K\p\\p\ = -S BG + S BG -(3U + f3U, (10) 

which can be cast into the form 

K\p\\p\=(3(F-F). (11) 

The free energy term is given as usual by F — U — SbgI 'P- The result above shows us that the physical meaning of 
the K-L entropy is nothing but the difference of the off-equilibrium and equilibrium free energies when the reference 
distribution is taken to be the equilibrium distribution given by Eq. (5) above. This result can be used, for example, 
to study equilibrium fluctuations or non-equilibrium relaxation of polymer chains [16]. 

III. SHARMA-MITTAL ENTROPY 

In this Section, we will briefly review the Sharma-Mittal entropy and some of its important properties from a 
generalized thermostatistical point of view. The Sharma-Mittal entropy [11] is given by 

SsM( P ) = Y ^- r [(^ P ^ ) - 1 l (12) 

i 

In the limit r^ 1, Sharma-Mittal entropy becomes Renyi entropy [2] which is 

Sfl(p) = ^-ln(I>?)' ( 13 ) 

i 

while for r^ q, it is Tsallis entropy [1] given by 

St( P ) = E fj g - ■ (14) 

In the limiting case when both parameters approach 1, we recover the ordinary Boltzmann-Gibbs (BG) entropy 
which reads 

Sbg(p) = -J>lnpi. (15) 

i 

For two statistically independent systems given by probability distributions p, and p' k , Sharma-Mittal entropy 
satisfies [9] 

S S M({PiP' k }) = S SM ({Pi}) + S SM ({p' k }) + (1 - r)S S M({Pi})SsM({p' k }). (16) 

This relation is important in order to see how Sharma-Mittal measure includes both Tsallis and Renyi entropies as its 
limiting cases. When we take r = q, it becomes the relation satisfied by Tsallis entropy and shows its nonextensivity. 
On the other hand, when r — 1, we have the extensive property which relates to Renyi entropy. In this sense, 
Sharma-Mittal entropy includes both extensive and nonextensive features in it. 

One feature of Sharma-Mittal entropy is that it fails to be concave [8] . The concavity entails thermodynamic stability 
and can be defined as follows: Consider probability distributions P = {pi,p2, ...,pn} and P' = {p[,P2, ■■■,p'n}- Let 
us also define an intermediate probability distribution given by P" = {p'{,P2, ■■■p'n} where 



Pi = m + (i - /"K,V/x g [o, i]. 



(17) 
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Now, S(P) is said to be concave if and only if 



S(P") > »S(P) + (1 - »)S(P'). (18) 

It is worth remark that BG entropy and Tsallis entropy is concave whereas Renyi entropy is not concave for all 
values of parameter q [17]. 

Another feature of Sharma-Mittal entropy is that it is not Lesche-stable [18]. Lesche stability checks the stability 
of the entropy functional under arbitrary small variations of the probabilities. It can be stated in a more rigorous 
way by defining the deformation of the probability distribution as 



\\P-P'\\ =£|Pi-Pil <S e ,V6 e >0. (19) 

i 

Then, S(P) is Lesche-stable if 



A = 



S{P) - S(P') 



< s,Ve > 0, 



(20) 



where S m ax is the maximum value that S can attain over all microstates. It has been shown that BG and Tsallis 
entropy is always Lesche-stable whereas Renyi entropy is unstable for all q ^ 1 [17, 18]. 

Lastly, it may be remarked that Sharma-Mittal entropy does not lead to finite entropy production per unit time 
whereas BG and Tsallis entropies do [8] . 



IV. SHARMA-MITTAL RELATIVE ENTROPY AND FREE ENERGY 



In order to study the physical meaning of Sharma-Mittal relative entropy in thermostatistical framework as we 
did with BG entropy in Section II, we have to obtain the equilibrium distribution corresponding to Sharma-Mittal 
entropy. For this purpose, we begin by maximizing the following associated functional 

1 w ( 1 ~ r ) w q 

i i Pi 

We take the derivative of the functional and equal it to zero in order to obtain the following 

= ^r- 1 ^ - a - p&o-^ -U) = 0, (22) 

Pi Q ^ 

where /?* is given by 

13* = , P (23) 
Multiplying Eq. (22) by pt and summing over the index i, we obtain 



(24) 



Note that tilde shows that the distribution is calculated at equilibrium. Substituting this explicit expression of a 
into Eq.(22), we calculate the associated equilibrium distribution [9] as 



Pi = ^ [l - (1 - q)f3**{e i - U)} ^ 



(25) 
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where (3** is given by 

8* 8 

8" = = 1^7"' (26) 

From Eq.(12), we see that 

(Ep') (H) = 1 + (1 - r),SsM ' (27) 

which also holds for equilibrium distribution given by Eq.(25) 

(Eft'J'^^i+a-fc (28) 

The Sharma-Mittal divergence is given by [19] 



/5Mbll^] = ^ T [(E^ 1 " < ') (U) - 1 



(29) 



We now substitute equilibrium distribution in Eq. (25) as the reference distribution into Sharma-Mittal divergence 
jiven above and obtain 

IsmIp II P] = ^ [{ X>?(1 + (1 - r)5 SM )^[l - (1 - - ^)]} (H) - l] • (30) 

i 

Using Eq.(27), it can be put into a more appropriate form which is 



IsmMpI = _2r { 1 + (j r \ SsM {l (l - g )/J«(I/ - C/)}(H)} - i]. 
r - 1 1 + (1 - r)5 SM 



(31) 



The expression above is very different than Eq. (11). It cannot be written in terms of free energy differences. 
Indeed, this can be achieved only by taking the limit q approaches 1 first 

IsmMpI = J_ [{ i + (i-^ e ^(^)((i-.) } _ 1]t (32) 

r_1 l + (l-r)SsM 

where the internal energy functions and /?** must be calculated at q = 1 and then considering the limit of the above 
expression as r goes to 1, which in turn gives 

K[p\\p\ = p(F - F). (33) 

This shows that the physical meaning of Sharma-Mittal divergence is the difference between the off-equilibrium 
free energy and equilibrium free energy when the reference distribution is taken to be the equilibrium distribution 
obtained from the maximization of Sharma-Mittal entropy only in the limiting case when both parameters approach 
to 1 in which case it approaches Kullback-Leibler entropy. 

This negative result above is the one exactly mimicked by the Renyi relative entropy [20]. It reads 

I q [p\\r] = -^- l HY.P>l q )- ( 34 ) 
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Note that this definition of Renyi relative entropy is always non-negative and equal to zero if and only if p = r. It 
also reduces to K-L entropy as the parameter q approaches 1. Let us write the associated functional where internal 

energy constraint is given in terms of escort probabilities i.e., U = , thereby yielding 



w 

w E £ iPl 



We again take the derivative of the functional and equate it to zero in order to obtain the following 

— = =^ -a-0 qp q \e l -U) = 0, (36) 



where /?* is given by 



Multiplying Eq. (36) by pi and summing over the index i, we find 

Q=T A_. (38) 

Note that tilde again denotes that the distribution is calculated at equilibrium. Substituting this explicit expression 
of a back into Eq. (36), Renyi equilibrium distribution reads 

= - V ~ tiF( Ei (39) 

e (i-q)b R 

Following the same steps as before, we then substitute equilibrium distribution above as the reference distribution 
into the Renyi relative entropy given by Eq. (34) and get 

q — 1 'r' e (i-9)i-n 
It can be put into a more appropriate form which is 

IMP\ = ^jH^ 1 -^-^ - {l-q)0"(U-U)), (41) 

where (3** is given by 

J 

Making Taylor series expansion about q = 1 for the exponential term within the parentheses and keeping the first 
two terms only, we have 

IMP! = "TIT Ml + ( J - ^)(^G - Sbg) - (1 - g)/3**((7 - C/)). (43) 
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Arranging the terms as follows 

I q [p\\p\ = Ml + (1 - 1){(S BG S bg ) (3**(U U)}} (44) 
and making Taylor expansion about q = 1 again but this time to the logarithmic term, we obtain 

IM\P\ = J^T^ 1 ?)[(^g - S BG ) ~ P(U - U)}, (45) 

which can be written as 

I q \p\\p\ = (3[(U - S BG IP) - (U - S BG /f3)]. (46) 
This is nothing but the free energy differences since it can be rewritten as 

I q \p\\p\=(3(F-F), (47) 

where free energy expressions are given by exactly as in the BG case. This is exactly the same expression obtained 
in Section II by using BG entropy and K-L entropy. It should be noted that the first Taylor expansion turned the 
Renyi entropies into BG entropies while second Taylor expansion turned the Lagrange multiplier and internal energy 
functions into their corresponding BG values [21]. 

This shows that Sharma-Mittal relative entropy behaves exactly in the same way as Renyi relative entropy when 
one considers them in terms of their physical meanings in a generalized thermostatistical framework. 

V. SHARMA-MITTAL RELATIVE ENTROPY AND SHORE-JOHNSON THEOREM 



At this point, it is important to remember Shore- Johnson theorem (see Refs. [22, 23] for details). According to it, 
any relative entropy J[p||r] with the prior and posterior which satisfies five very general axioms, must be of the 
form 



J\p\\r]=Y,Pih(p i /r i ), (48) 

i 

for some function h(x). These axioms are listed as 

1. Axiom of Uniqueness: If the same problem is solved twice, then the same answer is expected to result both 
times. 

2. Axiom of Invariance: The same answer is expected when the same problem is solved in two different coordinate 
systems, in which the posteriors in the two systems should be related by the coordinate transformation. 

3. Axiom of System Independence: It should not matter whether one accounts for independent information about 
independent systems separately in terms of their marginal distributions or in terms of the joint distribution. 

4. Axiom of Subset Independence: It should not matter whether one treats independent subsets of the states of 
the systems in terms of their separate conditional distributions or in terms of the joint distribution. 

5. Axiom of Expansibility: In the absence of new information, the prior should not be changed. 



Ordinary relative entropy i.e., K-L entropy is in accordance with Shore- Johnson theorem since we can find a function 
h(x) which allows us to write K-L entropy as Eq. (48) requires. This function h(x) is nothing but natural logarithm 
indeed. In the case of Renyi relative entropy [20] given by Eq. (34), we see that it cannot be cast into a form which 
will conform to the Shore- Johnson theorem for any function h(x). Inspection of Sharma-Mittal relative entropy shows 
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that it shares also this feature of Renyi relative entropy. In other words, both fails to conform to Shore- Johnson 
theorem. On the other hand, the noncxtcnsivc counterpart of relative entropy given by 

I Tsams Mr] = ^ [l _Y jP <lr]-% (49) 

i 

is seen to conform to Shore- Johnson theorem when the function h(x) is taken to be the the negative of the q- 
logarithm function defined by ln q (x) = x but with argument x replaced by 1/x [23]. Therefore, in the case of 
nonextensive thermostatistics, one has a relative entropy which conforms to Shore-Johnson theorem as K-L entropy 
in BG thermostatistics does. 



VI. RESULTS AND DISCUSSIONS 



The Sharma-Mittal entropy seems to generalize both Renyi and Tsallis entropies through the adjustment of its two 
parameters. We have investigated whether this interpretation is plausible from the thermostatistical point of view 
using the associated Sharma-Mittal relative entropy. The relative entropy is an important concept and has many 
applications in diverse fields such as quantum information theory [24], biophysics [12] and finance [13]. Its physical 
meaning in ordinary thermostatistics is the difference of free energies associated with equilibrium and off-equilibrium 
distributions. In this paper, we have shown that a similar result can be obtained in the case of Sharma-Mittal entropy 
but only when it reduces to K-L entropy, rendering the use of Sharma-Mittal relative entropy redundant in this 
generalized thermostatistical framework. We also observe that this is exactly how Renyi relative entropy behaves 
when it is subject to same kind of calculation. Another negative feature which Sharma-Mittal relative entropy has in 
common with Renyi entropy is that associated relative entropies violate the Shore- Johnson theorem which is satisfied 
by Kullback-Leibler entropy and one of the Tsallis relative entropies. Considering all these negative results common to 
both of them including the failure of concavity and stability, we believe that Sharma-Mittal entropy must be thought 
to be a step beyond not both Tsallis and Renyi entropies but rather as a generalization of Renyi entropy from a 
thermostatistical point of view although the explicit form of Sharma-Mittal entropy suggests that it has both of these 
entropies in its content when we only consider the limiting values of its parameters. 
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